Second-Order Learning Algorithm with Squared Penalty Term
نویسندگان
چکیده
منابع مشابه
Second-order Learning Algorithm with Squared Penalty Term
This article compares three penalty terms with respect to the efficiency of supervised learning, by using first- and second-order off-line learning algorithms and a first-order on-line algorithm. Our experiments showed that for a reasonably adequate penalty factor, the combination of the squared penalty term and the second-order learning algorithm drastically improves the convergence performanc...
متن کاملConvergence to a Second-order Point of a Trust-region Algorithm with a Nonmonotonic Penalty Parameter for Constrained Optimization Convergence to a Second-order Point of a Trust-region Algorithm with a Nonmonotonic Penalty Parameter for Constrained Optimization 1
A Abstract In a recent paper, the author (Ref. 1) proposed a trust-region algorithm for solving the problem of minimizing a non-linear function subject to a set of equality constraints. The main feature of the algorithm is that the penalty parameter in the merit function can be decreased whenever it is warranted. He studied the behavior of the penalty parameter and proved several global and loc...
متن کاملEnhanced Second Order Algorithm
Second order algorithms belong to the class of repetitive heuristic techniques, where a greedy heuristic is applied to different subsets of the space of feasible solutions of a problem. The algorithm establishes a sequence of self-contained solutions subspaces, defining an iterative process that guides a subordinate greedy heuristic through those subspaces. This steepest descent process establi...
متن کاملLearning Higher-Order Graph Structure with Features by Structure Penalty
In discrete undirected graphical models, the conditional independence of node labels Y is specified by the graph structure. We study the case where there is another input random vector X (e.g. observed features) such that the distribution P (Y | X) is determined by functions of X that characterize the (higher-order) interactions among the Y ’s. The main contribution of this paper is to learn th...
متن کاملFirst- and Second-Order Necessary Conditions Via Exact Penalty Functions
In this paper we study firstand second-order necessary conditions for nonlinear programming problems from the viewpoint of exact penalty functions. By applying the variational description of regular subgradients, we first establish necessary and sufficient conditions for a penalty term to be of KKT-type by using the regular subdifferential of the penalty term. In terms of the kernel of the subd...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Neural Computation
سال: 2000
ISSN: 0899-7667,1530-888X
DOI: 10.1162/089976600300015763